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Let Xi be members of a stationary sequence of zero mean gaussian random 
variables having correlations ExjXj = a 2 p u ~'\ < p < 1, a > 0. We address the 
behavior of the averaged product q m (p, a) = Ex\X 2 • • ■ X2m-iX 2m as m becomes 
large. Our principal result when a 2 = 1 is that this average approaches zero 
(infinity) as p is less (greater) than the critical value p c = 0.563007169 .... To 
obtain this we introduce a linear recurrence for the q m -(p, a), and then con- 
tinue generating an entire sequence of recurrences, where the (n + l)-st 
relation is a recurrence for the coefficients that appear in the nth relation. 
This leads to a new, simple continued fraction representation for the gener- 
ating function of the q m (p, o). The related problem with q m (p, a) = E\ 
Xi • ■ • x n \ is studied via integral equations and is shown to possess a smaller 
critical correlation value. 

I. INTRODUCTION 

The problem that we consider in this paper is as follows: Let {x,jf 
be a stationary sequence of zero mean, gaussian random variables with 
covariances 

Pij m ExiXj = a 2 p [H \ < p < 1, a > 0; i, j = 1, 2, ■ ■ • , (1) 

where E( • ) denotes mathematical expectation. What is the behavior 
of 



* Bell Laboratories. 



°Copyright 1983, American Telephone & Telegraph Company. Photo reproduction for 
noncommercial use is permitted without payment of royalty provided that each repro- 
duction is done without alteration and that the Journal reference and copyright notice 
are included on the first page. The title and abstract, but no other portions, of this 
paper may be copied or distributed royalty free by computer-based and other informa- 
tion-service systems without further permission. Permission to reproduce or republish 
any other portion of this paper must be obtained from the Editor. 



2993 



Qm(p, o) — E%\%2 ■•• *2m-l*2m (2) 

as m becomes large? 

In other words, the product in (2) is formed from samples of a gauss- 
markov process that are taken at regular intervals. Only an even 
number of samples is considered in (2) since an odd number would 
result in a zero average. 

Originally, the problem was conceived as a simple model for averages 
of multiplicative structures having infinite memory between the fac- 
tors of the product. Such products arise in the analysis of learning 
curves for many adaptive systems, and for these problems one encoun- 
ters products whose factors are noncommuting matrices. We felt that 
the analysis of a simple problem, such as that described above, would 
serve as a valuable guide to what results might be achievable for more 
realistic situations. However as one may readily imagine, as soon as 
the problem described in (1) and (2) was written down it became of 
interest in its own right, consisting as it does of a simple question 
about long familiar quantities. 

Our principal result is that for large m the behavior of the average 
product q m (p, a) in (2) depends on the relationship of p to a critical 
value, p c = p c (a). If p < p c , then q m (p, o) will approach zero exponen- 
tially fast; if p > pc, q m {p, <*) approaches infinity exponentially fast; 
finally, if p = p c , q m (p, a) -» qj^a). We find for a = 1, p c (l) = 
0.563007169391816 • • • , and g„(l) = 0.50900853 .... A plot of p c {a) is 
given in Fig. 1. All of these results were obtained from a continued 
fraction representation for the generating function 

QU, P,*)= l Qm(p, °)z m . (3) 

m=0 

Since q m {p, a) = a 2m q m (p, 1), we have 

Q(z, p, c) = Q(za 2 , p, 1), (4) 

so it is without loss of generality that we will set a = 1, Q(z, p) = Q 
(z, p, 1), and q m (p) = q m (p, 1). By introducing a sequence of generating 
functions, we show in Section II that 

Q(z,p)= 1 (5) 

1-P£ 

1 - 2p 3 2 



1 - 3p 5 2 



1 - "• 
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The value p[ a) is then the smallest p for which Q(a 2 , p) = oo, while the 
value q«,(a) is the limit as z —* 1 of (1 — z)Q(za 2 , p c ). 

Since methods are as interesting as results, Section III presents 
another approach involving integral equations for discussing the q m (p) 
behavior. Although this method is not rigorously justified for the 
present problem due to a non-hermitian kernel, it is applicable to a 



0.01 




Fig. 1 — Critical correlation value p e (<r 2 ) vs. variance a 2 . 
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related problem, the behavior of E \ x x • • • x m | as m -» oo [still assuming 
(1)]. Using the integral equation we show that p c , the critical value of 
p for this new problem, is strictly smaller than the p c defined above. 
This is of interest since it shows that the behavior of q m (p) is deter- 
mined both by how large \xi ■■■ x 2m \ is on the average, and by the 
extent of cancellation between positive and negative values of q m . 

Although we do not give the details here, it is not difficult to show 
that for all p < 1, q m (p) approaches zero with probability one as m 
becomes large. 

II. LINEAR RECURRENCES AND GENERATING FUNCTIONS 

Given 2m zero-mean jointly gaussian random variables x, of unit 
variance and correlations Ex t Xj = p,>, then a known formula 1 states 
that 

EXl ■■■ X 2 m= 2 PhhPisU ■ ' ' P*2m-l»2m» W 

all 
pairs 

where the unordered set |ii, • • • , km] is equal to the unordered set 
jl, 2, • • • , 2m}. The sum in (6) is over all distinct, unordered pairs of 
subscripts. That is, we do not count twice terms which differ only by 
interchanging the values within one or more subscript pairs, nor do 
we count twice terms which differ only by permuting subscript pairs. 
Thus there are (2m)!/(2 m m!) terms in the sum (6). 

If we denote permutations of 2m objects by a(i): i — > o(i), i = 1, 
2, • • • , 2m, then a succinct way of writing (6) when (1) holds is 

1 I \ a {2j)-c(2j-l)\ 

9„(P) = ^ I »'" • < 7 > 

I ml aeS^ 

the sum in (7) being over all (2m)! permutations of S 2m , the group of 
permutations of 2m symbols. Formula (7) shows immediately that 
q m ( P )>0ifp>0. 
Now define q (p) = 1 and write 

m 

qjp) = I b s (p)qm-s(p), m = 1, 2, • • • . (8) 

«=i 

We evaluate a few of the b s (p), writing for convenience fe,-(p) = 6,, 
<?«(p) = Qi- The evaluation is done from (8) by explicitly evaluating the 
q m (p) as needed. A partial list of 6,(p) follows: 

bi = p 

b 2 = 2p 4 

6 3 = 4p 7 + 6p 9 
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b 4 = 8p 10 + 24p 12 + 18p 14 + 24p 16 

6 5 = 16p 13 + 72p 15 + 108p 17 + 150p 19 + 144p 21 

+ 96p 23 + 120p 25 . (9) 

Equation (9) suggests the possibility that, for small p, only a few 
terms in (8) would need to be kept for an accurate description of q m (p). 
For example, keeping only one term yields 

q m = pq m -i, (10) 

or q m = p m . Since ExiX 2 = p, this approximation corresponds to treating 
the successive pairs of gaussian variables which determine q m (p), via 
(2), as independent. The next step after (10) would be to write 

q m = 6i<?m-i + b 2 q m -2. (11) 

This equation, involving b 2 as well, would be a correction to the 
"independence assumption," but one involving only up to fourth-order 
correlations, since, from (8) the highest average appearing in b 2 is 
E(xix 2 x 3 x 4 ). Further corrections are obtained by including more terms 
of (8), with higher order correlations entering. + 

Assuming the 6,-(p) to be known, the natural procedure would be to 
"solve" (8) using generating functions. We define these as follows: if 
yo> yi,y2, • • • is a bounded sequence of numbers, then the generating 
function, Y(z), of the sequence is defined for complex z, \z\ < 1, by 

Y(z) = l y,z'. (12) 

»=o 

Given Y(z), the y, are, in principle, uniquely determined. We assume 
that the reader is familiar with the use of generating functions. If not, 
consult Chapters XI and XIII of Feller. 2 

We define b (p) = 0, q (p) = 1, and call the generating functions of 
the bi{p), and g,(p) sequences B{z; p) and Q(z; p), respectively. The p 
dependence is explicitly indicated. 

If we multiply (8) by z m and sum from m = 1 to 00 (treating q m = 0, 
m < and b m = 0, m < 0), we obtain the basic relation 

Q(z; p) = z — h — :• (13) 

1 - B(z; p) 

Equation (13) thus allows us to determine, in principle, the q m from 
the b m . In particular, we have 



f The above interpretation prompts us to advocate consideration of the ideas repre- 
sented by (8) for analyzing more complex multiplicative structures, particularly when 
connections to some sort of independence approximation are a natural thing to seek. 
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I *M = QiU p) = 1 _ B 1 (1;p) . < 14 > 

and the critical value p c will be given by the equation 

B(l; Pc) = £ h(Pc) = I- (15) 

i 

Although we could work with the b m (p) themselves, a more conve- 
nient approach for finding p c numerically is to set up a continued 
fraction representation for the generating functions Q(z; p), or equiv- 
alently, B(z; p). It is this approach that we follow now. 

Recall (8) defining b s (p). Since these b s coefficients are a numerical 
sequence themselves, we can use the same reasoning that took us from 
the q k to the 6., and use it to suggest going from the b 8 to a new set of 
coefficients, 6 ( . 2) , via the following recurrence 



k 

k 



,<2> 
s=l 



bM = I b™(p)b k - s ( P ), k = 2,3,--., (16) 



where we define b (p) = 0. The recurrence (16) yields 

^^TMrVV (17) 

B {2) (z; p) being the generating function for the bf{p). To continue this 
procedure with a uniform notation, we define 

bi 1] (p) = b s (p) 

6i m, (p) = 0, m = l, 2, ... (18) 

and write 

&ir } (p) = i b™(pv>Z!M. m k : J; I] '. '. '. • (19) 

The corresponding sequence of generating functions are related by 

* V'N 1 - B {m+1) (z; P Y { } 

We use this repeatedly in (13) and obtain the continued fraction 
representation 1 ^ 



f The fact that this continued fraction does not terminate implies that Q(z; p) is not 
a rational function of z, and thus one cannot find a (finite-order) difference equation 
for the q m ( P ). See Ref. 3, Theorem 99.1, p. 400. 
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Qfe " )= ^^ (21) 

1 — b Y z 

1 - b[ 2) z 
1 - bfz 

1 - '• 

In (21) we have, for simplicity, written b\ m \p) = b[ m) . 

A relation which will be used later to aid in finding the b[ m) follows 
by setting k = 2 in (19), to obtain 

, <m+l), v _ b 2 (p) 

61 (p) - b™(p)' m 

We can calculate some of the b[ m \p) using the partial list of the 
bk(p) given in (9) to derive several b[ m) (p) from (19). Using (22) we 
then obtain 

b x - p 

b[ 2) = 2 P 3 

C = 3 P 5 

b[ 4) = 4p 7 

b[ 5) = 5p 9 . (23) 

The obvious guess that 

b[ m) = mp 2m -\ m = 1, 2, ■ . . (24) 

follows from a direct proof of the continued fraction given in the 
appendix. Assuming (24) to hold yields the simple representation 

Q(z;p)= (25) 

1 - pz_ 

1 - 2p 3 z 



1 - 3p 5 2 



1 - . 



The accurate numerical value 

p c = 0.563007169391816 ■ • • (26) 

was obtained by using this representation along with (14) and (15). 
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When p = p c , 



q„ = lim q k {p c ) = * • (27) 

M 1 kb k {p c ) 



Using the computed value of p c and the definition of the b k (p), we 
found numerically that 

q„ = 0.50900853 • • • . (28) 

It was quite surprising to us that Q(z, p) turned out to be a new, but 
simple, continued fraction. 

III. INTEGRAL EQUATION METHOD 

The purpose of this section is to introduce the integral equation 
method and to show that p c < p c , where p c is the critical correlation 
value for the related problem involving E \ Xi ■•■ x n \. 

We begin by developing an expression for Ex\ • • • x n . We have, from 
the Markov property of the x, sequence, 

EXl •■' X n - J •'• J X n p(Xn\Xn-l) 

• •• x l p{xi\xo)4>(x )dx ••• dx n , (29) 



where 



0(x) = 4= exp(-* 2 /2) (30) 

V27T 



is the standard normal density and 

p(y\ x) = ] exp[-(y - px) 2 /2(l - P 2 )] (31) 

V27r(l - p ) 

is the generic form of the conditional densities occurring in (29). 
Define a kernel K(x, y) by 

K(x,y) =yp(y\x), 



= r 



Kf(x)= K(x,y)f(y)dy. 

Then (29) may be written in the inner product notation of Hilbert 
Space 

E Xl • • • Xn = (K n l, 0), (32) 

where is the normal density (30), 1 is the unit constant function, 
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and K n is the nth iterated kernel. Now assume, heuristically, that K" 
has the usual expansion 



K n (x,y)= I \Wj(x)+j(y) 



(33) 



in terms of eigenfunctions \J/j(x) and eigenvalues A, of K. Then 
Ex\ • • • x n would remain bounded, if, and only if, the largest eigenvalue 
Xi = Xi(p) is less or equal to one; thus \\{p c ) = 1 would determine p c . 
Unfortunately there is no general eigenexpansion theory available for 
K since it is not symmetric and is not symmetrizable. 

Fortunately symmetry holds for the integral equation method when 
one expresses E\ Xi • • • x„\ via kernels. Define, in analogy to (32), 



If we further define 



where 



K(x, y) = \y\p(y\x). 
h(x) „ t 



J(x, y) = 



K(x, y), 



then 



J(x, y) = 



h(y) 
h(x) = J\x~\exp(-x 2 /4), 

1 1 + p 2 



V27r(l - p 2 ) 
pxy 



V|ry|exp 



4 1 - p* 



(x 2 + y 2 ) 



exp 



1-P S 



is a symmetric kernel. 
As in (29), 



E\xi ••• *„| - J • • • J K(x n - U x n )R(x n - 2 , x„-i) 
• • • K(x , xi) <t>(x )dx • • • dx n 



= J • • J dx ■■■ dx n 



J{Xq,Xi) 



• • • J(X n -i, XJ-— ~ 0(X O ) 
h(Xo) 



= [J%-). 



(34) 



(35) 



(36) 



(37) 



(38) 
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Since J is symmetric and square-integrable, it is a Hilbert- Schmidt 
kernel and so has a discrete spectrum. Further its maximum eigen- 
value, X, is given by 

X = sup — . (39) 

Since J(x, y) > 0, we see that the maximum eigenfunction g = g{x) is 
nonnegative and X > as well. Further since h and <p/h are nonnega- 
tive, (h, g)>0 and (0/h, g) > so that E | *i • • • x n j - (J"h, 4>/h) -> 
oo if and only if X > 1. 
Define /„ by 

f a {x) = VMexp(-ax 2 /4), (40) 

and note that from (39), 

X > (Jf a , f a )/(fa, fa). (41) 



Now 



(/«,/«) = J |*|exp(-ax 2 /2)dx = 2/a (42) 



and 

V27r(l - p z ) •'-« • / -» 



where 



cxp | - | (x 2 + y 2 ) + j^jJ dxdy, (43) 



-K^H- 



C = T 



Set y = xu and integrate over x to obtain 

4 | \u\du 

{Jfc " fa) = V2tt(1 - p 2 ) J- (c(l + u 2 ) - ^) 2 ' 

P = A- («> 

1 - P 

Using Ref. 4 (p. 68, 2.175) we evaluate the last integral as 
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(Jfa, h) = 



4 4/3 _. fi 

i s- — I — tf> tan -7= 

V2tt(i - p 2 ) La a 3/2 Va. 



(A = 4c 2 - f. (46) 



Setting a = 0.5, p = 0.55, we find that c = 1.18369, = 1.57706, A = 
3.11738, (J/„, /„) = 4.0824, (f a , /„) = 4, so that A > 1.012. Thus for 
p = 0.55, £ I Xi ■ • • X n I — > °o, and /o c < 0.55. We have seen that p c > 
0.563 so the claim is proven. 
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APPENDIX 

Combinatorial Derivation of Continued Fraction 

In this appendix we give a direct combinatorial proof of the contin- 
ued fraction representation (25) of Q(z, p). This derivation is complete 
in itself, but we preferred the method of the text for showing where 
the continued fraction comes from. Our starting point is the formula 
(7). Let us define, for a £ S 2m , 

m 

V(o)= I \<r(2i) -<r(2i-l)|. (47) 

1=1 

For 1 ^ k ^ m, let 

S(m, k) = \a G S 2m :ai,2m) = 2m, 
a(2m - 2) = 2m - 1, 
<r(2m - 4) = 2m - 2, • • • , 

<r(2m - 2k + 2) = 2m - k + 1\. (48) 

For k = 0, we adopt the convention that S(m, 0) = S 2m . We also define 

so that u(m, 0) = q m . (We take u(0, 0) = qo = 1, and u(m, k) = for 
k < and k > m.) Our key result is: 

Lemma. Ifm>l,k>0, then 
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u(m, k) = k P 2k - x u(m - 1, k - 1) + u(m, k + 1). (50) 

Proof. We will prove this for 1 « k < m - 1, as the other cases are 
easy. Let 

S' = |ffG S(m, fe):2m - fc 

G {<r(2m - 1), <r(2m - 3), • ■ • , <r(2m - 2k + 1)}}, 

S" = Sim, k) - S'. (51) 

If a G S", we construct a permutation a* G S(m, k + 1) by changing 
the action of <x on four letters in such a way that V((r) = Via*) and 
<x*(2m - 2k) = 2m - k. To define a* precisely, letp and r be such that 
\r, 2m - k\ = \a{2p - 1), <x(2p)j. Then, if we associate to a the vector 
Ma) = (<r(l), <t{2), ■ • • , a(2m)), the vector A(<r*) is obtained from A (a) 
by interchanging the pairs \a(2m -2k- 1), a{2m - k)\ and {r, 2m - 
k\ so as to keep the same ordering in the first pair, but possibly 
reversing it in the second, so as to have o*(2m - 2k) = 2m - k. As an 
example, if m - 5, k = 3, and A(a) = (7, 2, 4, 1, 6, 8, 3, 9, 5, 10), then 
A( *) = (4, 1, 2, 7, 6, 8, 3, 9, 5, 10). It is clear that o* G S(m, k + 1) 
and V(a*) = V(<r). Moreover, every t G S(m, fe + 1) can be represented 
in exactly 2(m - k) ways as r = a*, a* G S". Therefore, 

I p V( "> = u(m, k + 1). (52) 



2"-*(m - k)\ a £h> 

Suppose now that aES'. Then 2m- k = <r(2m - 2r + 1) for some 
r, 1 s* r ^ k. We now define a permutation a' G S(m — 1, k — 1) as 
follows: In A(<r), delete a = a(2m - 2r + 1)(= 2m - k) and b = o 
• (2m - 2r + 2) and reduce the remaining entries that are between a 
and b by 1, and those that are larger than max(a, 6) = a by 2. As an 
example, if m = 5, k = 3, and A(a) = (2, 1, 6, 3, 5, 8, 7, 9, 4, 10), then 
A(o') = (2, 1, 6, 3, 5, 7, 4, 8). The resulting vector clearly equals A(a') 
for some a' G S(m - 1, k - 1), and each t G S(m - 1, k - 1) has 
exactly k such representations. Further, V(a) equals the sum of 
(i) V{a'), (ii) a - b for the pair that was dropped, (hi) 2 for each of the 
r - 1 pairs (<r(2m - 2/ + 1), <r(2m - 2; + 2)) for 1 < ; < r - 1, since 
in each such pair <r(2m - 2; + 2) > a, <r(2m - 2] + 1) < b, and finally 
(iv) 1 for each of the k - r pairs (cr(2m - 2j + 1), <r(2m - 2; + 2)), r + 
^j^k, since in each of them a(2m - j + 1) <b,b< a(2m - 2j + 2) 
< a. Hence, 

V(a) = V(<r') + a - b + 2(r - 1) + k - r. (53) 

But a = 2m - k and 6 = <r(2m - 2r + 2) = 2m - r + 1 from the 
definitions of Sim, k), so 

V(<t) = Via') + 2fc - 1. (54) 
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Hence, we have 
1 



2 P VM = kp 2k - l u{m -l,k- 1), (55) 



2 m "*(m - k)\ „£■ 

which proves the lemma. 

We now can use the recurrence of the Lemma to derive the continued 
fraction expansion of the generating function. Let 

fk = fk(z) = l "(m, k)z m , k = 0, 1, • • • , 

771=0 

which for the moment we regard as formal power series in z. Then the 
Lemma gives us 

h - /o " 1, (56) 

and for k>2, 

fk = l u(m, k - \)z m - (k - l)p 2 *" 3 J) u(m - 1, k - 2)z m 

m m 

= A_! - (k - l)p 2 *- 3 z/*-2. (57) 

Relations (56) and (57) show that for k ^ 0, 

fk = s k fo - r k , (58) 

where s = Si = 1, r = 0, r x = 1, and for k 2* 2 both s k and r k satisfy 
the recurrence 

x k = x k -i - (k- l)p 2k - 3 zx k - 2 . 

Hence the quotients r k /sk are the partial quotients of the continued 
fraction R(z, p) on the right side of (25), and s k and r k converge as 
k — » oo to power series (in z) s(z, p) and r(z, p), respectively, for which 

fife „) = !&4 (59) 

s(2, p) 

On the other hand, since fk starts with a term involving Zk, we conclude 
that f k converges to in the ring of formal power series as k — * oo. 
Therefore, from (58), 

fo = r -!fiA = R{Zy ). m 

s(z, p) 

Since f Q = Q(z, p), we obtain the relation (25), at least in the ring of 
formal power series in z. However, the continued fraction (25) is 
clearly a meromorphic function of z for p fixed, < p < 1, and it is 
analytic at 0. Hence (25) holds as an equality among meromorphic 
functions, and we can obtain from this the exponential decrease of the 
q m (p) for p < p c and the exponential increase for p > p c . 
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